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! hile the United States 
has made strides in 
increasing college access 

\ ao | among low-income 
students, college completion has remained 
low. Graduation rates are particularly low 
at the nation’s community colleges,’ which 
enroll a disproportionate percentage of low- 
income and nontraditional college students.” 
Only 20 percent of full-time, first-time, 
degree-seeking students at public two-year 
colleges earn degrees within three years. 

Seeking to address this problem, in 
2014 three community colleges in Ohio — 
Cincinnati State Technical and Community 
College, Cuyahoga Community College, 
and Lorain County Community College — 
undertook a new strategy to help more of 
their lowest-performing students succeed 
academically. The highly successful 
Accelerated Study in Associate Programs 
(ASAP) developed by the City University of 
New York (CUNY) provided a model. 

ASAP is a comprehensive program that 
provides students with up to three years 
of financial and academic support and 
other support services to address multiple 
barriers to student success, with the goal 
of helping more students graduate within 
three years. MDRC’s random assignment 
evaluation of CUNY ASAP found that after 
three years, 40 percent of ASAP students 
graduated compared with just 22 percent of 
control group students. After six years, ASAP 
students continued to outperform the control 
group, with 51 percent of the program group 
earning degrees compared with 41 percent of 
the control group.‘ 

The Ohio programs are based closely 


on the CUNY ASAP program with a few 
adjustments. Like the CUNY ASAP model, 
the Ohio program model as designed 
required students to enroll full time and 
encouraged them to take developmental 
(remedial) courses immediately; provided 
comprehensive support services such 
as intensive advising; provided financial 
support to help students meet participation 
requirements; and offered blocked courses 
and condensed schedules. See Box 1 for 
a list of program components with some 
definitions of terms.5 

This brief presents academic effects after 
two years for the pooled, full study sample 
in the ASAP Ohio demonstration.® The 
findings show that students in the program 
group clearly outperformed the control group 
with respect to persistence in school, credit 
accumulation, and graduation. Graduation 
rates more than doubled: 19 percent of 
the program group earned a degree or 
credential after two years compared with 8 
percent of the control group. The brief also 
presents some findings from analyses of the 
programs’ implementation and costs. 


OUT THE 
L\LUATION 

The demonstration launched in 2014 

and included the three Ohio colleges, 
CUNY, MDRC, and the Ohio Department 
of Higher Education.’ The three colleges 
began operating their own programs in 
2015 based on the CUNY ASAP model: 
CState Accelerate at Cincinnati State, 
Degree in Three at Cuyahoga Community 
College, and Students Accelerating in 
Learning at Lorain County Community 
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BOX 1: SUMMARY OF OHIO PROGRAM MODEL COMPONENTS 


Student Support Requirements and Messages 
* Enhanced advising * Full-time and summer enrollment 


* Enhanced career- * Taking developmental education courses early 


development services 


* Enhanced tutoring 


* Graduating within three years 


Financial Support Course Enrollment 


* Tuition waiver iy Wretoyaksxo) (0 F-1x-ve Msxel aL <Yo[B] (-WeValem ») (eel <Yo Meco) | g-5< 


* Textbook assistance * First-year seminar* 


* Monthly incentive* 

Program Management 

* Managed locally within each college 
* Dedicated staffing 


NOTES: *The monthly incentive of $50 (in the form of a grocery/gas card) is for meeting advising, tutoring, and career-development-service requirements. 


Blocked courses are seats held in specific courses to allow for the condensed schedule. 


‘The first-year seminar is a student success course taken in the first semester, designed to introduce new students to strategies for being successful at college that could 


include goal setting, study skills, and career and academic planning. 


College.’ The Ascendium Education Group 
provided anchor funding, supplemented 
by grants from a consortium of other 
philanthropies, including the Bill & Melinda 
Gates Foundation, the ECMC Foundation, 
the Ford Foundation, the Greater Cincinnati 
Foundation, Haile U.S. Bank Foundation, 
KnowledgeWorks, the Kresge Foundation, 
the Laura and John Arnold Foundation, and 
the Lumina Foundation. CUNY provided 
in-depth technical assistance, while the Ohio 
Department of Higher Education coordinated 
knowledge sharing among college leaders and 
program staff members at the three colleges. 
MDRC provided operational support, led the 
evaluation, and oversaw the demonstration. 
The evaluation of the Ohio programs uses 
random assignment — a lottery-like process — 
to place interested, eligible students into either a 
program group eligible to receive the programs’ 
services and benefits or a control group who 
receives regular services. The differences 
between these two groups’ outcomes represent 
the estimated effects of the opportunity 
to participate in these programs. Random 
assignment ensures that student characteristics 
are not systematically different at the start of the 
study, allowing differences in later outcomes to 
be attributed to the new program. 


THE STUDENTS IN THE STUDY 


Students who met the following criteria were 
eligible for the study: 


« They were from low-income families (that is, 
they were eligible for Pell Grants). 


* They were seeking degrees. 
* They were willing to attend full time. 


* They were majoring in degree programs that 
could be completed within three years. 


New students were eligible for the study, 
as were continuing students with up to 24 
credits. Students were enrolled into the study 
just before the start of the spring 2015, fall 
2015, and spring 2016 semesters, for a total 
of 1,501 students (806 in the program group 
and 695 in the control group). As part of the 
study intake process, students completed a 
baseline information form that captured their 
demographic information. Roughly half of 
the students in the sample are considered 
nontraditional.? Approximately three in four 
had at least one developmental education 
course requirement at the time of random 
assignment. About 60 percent were employed 
when they entered the study, with about one- 
fourth of those working full time. For a full 
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table of baseline characteristics, see Appendix 
A. (All of the supplementary appendixes are 
online only.) 


IMPLEMENTATION 
FINDINGS 

This brief presents findings on how each of 
the program components was implemented.'° 
Data sources include interviews with program 
staff members, administrators, and students; 
program records; and college records. Together 
these data show that across the three colleges, 
most of the components were implemented as 
planned. While all colleges had advising and 
financial support in place as expected, they 

all struggled in getting students to make full 
use of the career development services and 
tutoring, and in implementing blocked courses 
as planned. Over time, each college adjusted 
its approach to delivering career development 
services and tutoring, in particular, to ensure 
students had various ways to get those forms 
of support. More detail on the implementation 
of each component follows. For tables 
presenting program components and 
participation data, see Appendix B. 


REQUIREMENTS AND MESSAGES 

Students in the program were required to enroll 
full time in the fall and spring semesters, and 
summer attendance was encouraged. According 
to interviews with program staff members and 
students, program advisers emphasized these 
messages. Over 80 percent of students who 
enrolled, enrolled full time in their first two 
semesters, though this proportion dropped 

to 57 percent by the fourth semester. Program 
advisers could continue to support students 
who did not comply with the enrollment 
mandate, although these students were not 
eligible for monthly incentives. Program 
advisers encouraged students to take their 
developmental courses as soon as possible 
and to aim to graduate within three years; they 
helped students reach these goals by planning 
their schedules with them. About two-thirds of 
students needing developmental education were 
enrolled in such courses in the first semester. 


STUDENT SUPPORT SERVICES 

Student support services are the heart of the 
Ohio programs and participation in these 
services earns students a monthly incentive. 
Of the three components of student services, 
advising was implemented with the highest 
fidelity to the program model and had the 
highest participation rate among program 
students. 


Advising 

Students are required to meet with their 
program advisers twice a month in the 
first semester. Starting in the second 
semester and extending through the 
end of the three-year program, advisers 


“They really look 


out for you ... it’s 


sort students into low-, medium-, and like if you do your 
high-need groups, for which the advising 
requirements differ. Program advisers, part they’ll do theirs 
many of whom already had advising : 
experience at the colleges, have low so that definitely 
caseloads (typically around 125 students). helped. ” 


Advising was implemented as expected 
across the three colleges, with caseload 

sizes as planned. Ninety-five percent 

of enrolled program students met with 

their Ohio program advisers at least once 
during the first semester; more than 60 percent 
met with their advisers six or more times. In 
later semesters high percentages of students 
continued to meet with advisers at least once, 
while on average each student met with an 
adviser less often. This decline in the average 
number of meetings was expected once 
students were separated into groups based on 
need, since the medium- and low-need groups 
were required to meet with advisers less often. 
Program students who were interviewed for 
the study credited the advising more than any 
other program component for their success 
and persistence in college. See Box 2 for 

one student’s explanation of her program 
experience and all the ways her adviser 
provided support. 


Career Development Services 
All program students were required to 
complete one career-services activity per 


In her 40s, Sally had been out of school since graduating from high 
school 25 years ago. She was a stay-at-home mom to her children, 
who were in their 20s with their own children when she returned 
to college. Sally had tried finding a job, but none paid a wage high 
enough for her to support herself so she decided to go college. She 
joined the Ohio program in spring 2016 and graduated in spring 
2018. Sally describes the wide-ranging support she received from 
her Ohio program adviser: scheduling classes, picking out books, 
going to the food bank together, sharing contact information for 
personal resources, and other forms of help. The financial support 
— especially the money for books and the monthly incentives — 
was very helpful to her since she relied on financial aid for daily 
living expenses. Sally describes herself as “clueless” when she 
started school. “When | first started | had no clue because | was 
out of school for 25 years. So this was a big adjustment. And now 
I'm rolling right into it. | know what I’m doing.” However, she 
worries about how she'll manage without her adviser when she 
begins classes at a four-year university. 


semester. Approved activities took a variety 


of forms and the number of options grew as 
the colleges struggled to get students to fulfill 
this requirement. Options included meeting 
individually with a career services staff member; 
taking an online career assessment; engaging 


in an internship experience; or attending a 
résumé-building, interviewing-skills, or job- 
search workshop. All of these options were 
also available to students in the control 
group. It nevertheless remained difficult to 


get all students to participate in this program 
component, although participation improved 
over time. (Participation ranged from 45 percent 


to 69 percent over the first four semesters.) 


Tutoring 


Program students in developmental education 
courses were required to attend at least three 
hours of tutoring per month.” From the start, 
it was a challenge to get students to participate 
in tutoring as required. Students and program 


advisers noted that many students had 
multiple responsibilities outside of school 


(such as work or family commitments) and 


found it hard to make time for tutoring. In 
response, the colleges allowed students 
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several options to fulfill the requirement: using 
on-campus tutoring resources, accessing 
online tutoring, working with an instructor, 
using a peer or family member, or using a tutor 
exclusively available to the program students. 
Expanded options helped the schools increase 
participation rates among students enrolled in 
developmental education from 58 percent in 
the first semester of the program to 72 percent 
in the second. 


Course ENROLLMENT 

The programs planned to help students 

with course enrollment by offering priority 
registration (meaning program students were 
able to register earlier than other students) 

and formal blocked courses to ensure program 
students took courses together. While priority 
registration was implemented as planned, 

all the colleges struggled to implement the 
planned approach to blocked courses and 

the first-year seminar, in large part because 
students had competing demands and 
complicated scheduling needs. Rather than 
working with the registrar to reserve seats in 
specific sections, program advisers encouraged 
program students to enroll in certain sections 
during an early-registration period. 


FINANCIAL SUPPORT 

As planned, all program students whose 
financial aid packages did not fully cover 
tuition and fees were granted waivers that 
covered the difference. Program students 
also received financial assistance to cover the 
costs of textbooks at the campus bookstore 
(at least $300 during each of the fall and 
spring semesters and about half as much 
for summer semesters)."* These financial- 
support components were implemented as 
expected across the three colleges. 

For the monthly $50 incentive, each 
campus disbursed gift cards from stores in its 
area where students could purchase groceries 
or gas. This component was implemented as 
planned. Among students who stayed enrolled 
in college, at least 45 percent received three or 
more payments in each semester. 
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PROGRAM MANAGEMENT the Ohio Department of Higher Education. 

A program director at each college managed These meetings allowed college leaders and 
the program advisers and reported to the program staff members the opportunity to 
provost or a vice president. In these first connect with each other and the CUNY ASAP 
two years of operations, the advisers and technical assistance team, discuss their 
program directors were fully dedicated to progress toward important benchmarks, and 
the program. Of the three colleges in the solve problems together. Program advisers 
study, one experienced little staff turnover, sometimes also had separate conversations to 
one experienced a modest amount, and one share their experiences and exchange advice. 


experienced a significant amount. 


Each college used a data-management THE PROGRAMS’ 


system to record program participation EFFECTS ON 
and generate reports about participation ACADEMIC OUTCOMES 
benchmarks that the program staffand senior After two years, the program group in this 
college staff reviewed monthly. Each college study clearly outperformed the control group 
also used its own enrollment and graduation with respect to persistence in school, credit 
records to track student progress toward accumulation, and graduation. 
program goals each semester and annually. 

Collectively, the three colleges convened IMPROVED PERSISTENCE 
either in person or over the phone at least Figure 1 displays enrollment at any 
quarterly, in meetings often facilitated by postsecondary institution and full-time 


FIGURE 1. OHIO PROGRAMS BOOST ENROLLMENT 
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SOURCE: MDRC calculations using data from the National Student Clearinghouse and transcript data from the demonstration colleges. 
NOTES: Estimates are for the full sample of 1,501 students. 
Enrollment is based on all available data and combines spring and summer enrollment. 
FT = full-time, defined as enrollment in 12 or more credits and based on data from the college of random assignment only. 
A two-tailed t-test was applied to differences between research groups. Statistical significance levels are indicated as: *** = 1 percent; ** =5 percent; * = 10 percent. 
Estimates are adjusted by site, cohort, gender, race/ethnicity, age, parental status, marital status, weekly hours worked, dependence on parents for 50 percent or 
more of financial support, whether a student is the first family member to attend college, whether a student earned a high school diploma, the number of developmental 
education requirements at the time of random assignment, and intended enrollment level. 
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FIGURE 2. OHIO PROGRAMS INCREASE CREDIT ACCUMULATION 
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SOURCE: MDRC calculations using transcript data from the demonstration colleges. 
NOTES: Estimates are for the full sample of 1,501 students. 
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Credits earned in spring and summer semesters are combined. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are indicated as: *** =1 percent; ** = 5 percent; 
* = 10 percent. 

Estimates are adjusted by site, cohort, gender, race/ethnicity, age, parental status, marital status, weekly hours worked, dependence on parents for 50 
percent or more of financial support, whether a student is the first family member to attend college, whether a student earned a high school diploma, the 


number of developmental education requirements at the time of random assignment, and intended enrollment level. 


enrollment rates at the study colleges 
during the first two years after students were 
randomly assigned. Beginning in the first 
semester, there is a statistically significant 
estimated effect on enrollment of 

4 percentage points. The effect on 
enrollment grows to 12 percentage points 

in the second semester and remains above 
9 percentage points through the end of the 
four-semester follow-up period. 

The effect on full-time enrollment, a 
requirement of the program, is even larger. 
In the first semester, there is an estimated 
18 percentage point effect on full-time 
enrollment. The effect on this measure 
remains large and significant throughout the 
rest of the follow-up period, ranging from 11 
percentage points to 19 percentage points. 
This finding shows that there is a sizable 
group of students who currently enroll part 
time but would enroll full time with the right 
requirements and support. 


The program advisers actively encouraged 
program group students to enroll in courses 
during the summer, an activity that increases 
momentum and has been correlated with 
college completion.’ Summer course tuition 
was covered and monthly incentives were 
available to students during this time. There 
was an estimated 24 percentage point 
increase in summer course enrollment during 
the first summer after random assignment 
(an increase from 31 percent to 55 percent), 
and a 12 percentage point increase during 
the second summer (from 23 percent to 35 
percent; see Appendix Table C.1). 


INCREASED CREDIT ACCUMULATION 

Figure 2 represents cumulative total credits 
earned (both developmental and college-level) 
during the first two years after students were 
randomly assigned.’* The program group 
earned roughly two credits more than the 
control group per semester, for a total increase 
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FIGURE 3. OHIO PROGRAMS INCREASE GRADUATION 
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SOURCE: MDRC calculations using data from the National Student Clearinghouse and transcript data from the demonstration colleges. 


NOTES: Estimates are for the full sample of 1,501 students. 


Degrees earned in spring and summer semesters are combined. 
A two-tailed t-test was applied to differences between research groups. Statistical significance levels are indicated as: *** =1 percent; ** =5 percent; 


* = 10 percent. 


Estimates are adjusted by site, cohort, gender, race/ethnicity, age, parental status, marital status, weekly hours worked, dependence on parents for 50 
percent or more of financial support, whether a student is the first family member to attend college, whether a student earned a high school diploma, the 
number of developmental education requirements at the time of random assignment, and intended enrollment level. 


of eight credits at the end of four semesters. 
This effect represents a 37 percent increase in 
credits earned after two years. To the authors’ 
knowledge, this is one of the largest increases 
in credit accumulation — an important 
indicator of academic progress — that has 
been observed in a rigorous evaluation. 


INCREASED GRADUATION RATES 

As shown in Figure 3, after two years (or 

four semesters), 19 percent of the program 
group had earned a degree or credential, 
compared with 8 percent of the control 

group, a Statistically significant increase 

of 11 percentage points. This effect means 
that two-year graduation rates more than 
doubled (increasing by 140 percent). Nearly all 
graduates earned associate’s degrees. 


THE EFFECTS OF THE OHIO PROGRAMS 
ComPARED WITH CUNY ASAP 

CUNY ASAP achieved dramatic effects on 
academic outcomes including persistence 
in college, credit accumulation, and degree 
completion. A comparison of the two-year 


results from the Ohio evaluation with MDRC’s 
evaluation of CUNY ASAP shows that the 
Ohio programs were largely able to achieve the 
same net effects as CUNY ASAP, and in some 
cases were able to exceed them."> The Ohio 
programs’ effect on credits earned after two 
years (8.1 credits) is similar to CUNY ASAP’s 
effect (7.6 credits). The effect on degrees 
earned after two years is larger in Ohio (11 
percentage points compared with 6 percentage 
points at CUNY). 

The overall outcome levels on measures 
are different in Ohio than they were in the 
CUNY ASAP evaluation, which may be a result 
of differences in the types of students served 
in the two evaluations, as well as the different 
structures, resources, and services available 
to students outside the program. The Ohio 
colleges and CUNY have different political 
leadership, governance structures (CUNY is 
centralized and the Ohio programs operate 
at three different colleges that are not), and 
student populations, as roughly half of the 
students in the Ohio study were nontraditional 
compared with about a third at CUNY. 
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FIGURE 4. PROGRAM COMPONENT COSTS (PERCENTAGES OF THE TOTAL) 
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Errects AMONG SUBGROUPS 

In addition to estimating the overall 

average effect of the Ohio programs, the 
study measured whether the program was 
effective for various types of students, 

or subgroups. Of special note, the study 
confirms that the Ohio programs were 
effective for students who were “college- 
ready” and for students who were required 
to take developmental education courses.’® 
Also explored were effects for the three study 
colleges; for students of different genders, 
races, ethnicities, and ages; for students 
who had and had not earned high school 
diplomas before enrolling in the study; and 
for traditional and nontraditional students. 
The Ohio programs had large positive 
effects for all the subgroups examined, as 
shown in Appendix C. MDRC’s evaluation of 
CUNY ASAP also found that ASAP improved 
academic outcomes for a range of students. 
Together, these findings provide strong 
evidence that this program can be effective 
for many types of students, including groups 
that traditionally have low success rates. 
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THE COSTS OF THE 
PROGRAMS 

Figure 4 breaks down the direct costs 

of the Ohio programs, which include 


administration and staffing, student services, 


and financial support.” The total annual 
direct cost per program group member is 
$2,331 (a total that includes program group 
members who did not enroll). About 42 
percent of the direct cost of the program 
comes from administration and staffing, 
mostly from senior leaders and the fully 
dedicated program directors who managed 
the program and provided quality control. 
This percentage is high in part because the 
program was small. Administration would 
probably be a smaller proportion of the total 
cost if the program were larger. 

Financial support — including the 
monthly incentives, textbook subsidies, 
and tuition assistance provided to program 
students — makes up 32 percent of the 
programs’ cost. Textbooks are the biggest 
expense in this category, accounting for just 
over a third of the costs, followed by the 


100% 
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tuition waivers. Tuition waivers, accounting 
for $254 per program student on average, 
were applied to tuition not covered by Pell 
Grants or state aid, and most students did 
have Pell funds because Pell Grant eligibility 
was an eligibility requirement. Targeting the 
program in this way enabled the colleges 

to keep this cost down. Monthly incentives 
averaged $210 per program student per year. 
(A student could earn three to four monthly 
incentives per semester, but the average 
includes students who did not enroll or 
complete all requirements.) 

Finally, about 26 percent of the direct 
cost of the program comes from the student 
services provided, mostly from the fully 
dedicated advisers. Tutoring and career 
services costs are quite minimal, probably 
because the colleges were able to use their 
existing resources in these areas. 

These costs are much lower than the 
those found in MDRC’s evaluation of 
CUNY ASAP, in part because salaries are 
lower in Ohio, in part because of the lack of 
blocked or linked courses (which had costs 
associated with them in the CUNY ASAP 
evaluation), and in part because the Ohio 
colleges’ monthly incentives were valued 
at about half of CUNY’s MetroCards (used 
for travel in New York City’s mass transit 
system). The Ohio colleges also used fewer 
advisers to serve an equivalent number of 
students than CUNY’s original model, and 
may have further reduced costs by using 
existing career services and tutoring.” 

Appendix D provides additional cost 
calculations, including base costs, indirect 
or induced costs, and net costs, as well as 
alternate calculations of direct costs (see 
the appendix for definitions of all these 
terms). All cost estimates are based on data 
through the end of 2017. The final report will 
include an additional year of cost data and 
will present a cost-effectiveness analysis, 
comparing the cost per graduate in the 
program and control groups. 


CONCLUSION AND 
NEXT STEPS 


The three colleges in Ohio were largely able 
to implement programs based on CUNY 
ASAP, and these programs dramatically 
improved students’ academic outcomes over 
two years. Thus far, they more than doubled 
graduation rates. 

Historically, attempts to replicate effective 
programs find that replications rarely achieve 
results similar to the original.'? The Ohio 
demonstration has been an exception. The 
leadership and commitment of the three 
colleges, the Ohio Department of Higher 
Education, and the Ohio Governor’s Office 
contributed to the successful results, as did 
the strong technical assistance provided by 
CUNY. Additionally, the Ohio demonstration 
provides evidence that the model can work in 
a different context and with a different student 
population, as many more students in Ohio 
were nontraditional. These findings further 
validate the effectiveness of the CUNY ASAP 
model and add to the growing body of evidence 
on effective strategies for improving the 
educational outcomes of low-income students. 

Much has been learned from these three 
colleges’ experiences implementing their 
programs. Additional colleges — Westchester 
County Community College in New York and 
Skyline College in California — have begun 
implementing similar programs with technical 
assistance from CUNY. Meanwhile, in Ohio, 
one of the three colleges in the current study 
is sustaining and expanding its program with 
a goal of making it available to most eligible 
students in the coming years. A second 
college is still considering how to sustain the 
program and meet the financial requirements 
needed for implementation. And the third 
college is taking lessons from its experience 
with this program and embedding them into 
other broad-reaching programs and policies. 

MDRC’s evaluation will continue 
tracking longer-term academic data. A future 
report will present effects after three years, 

a cost-effectiveness analysis, and the full 
implementation story. 


NOTES 


1 The three-year graduation rate at two-year, degree-grant- 
ing institutions averages 30 percent nationally, while the 

six-year graduation rate at four-year, degree-granting insti- 
tutions averages 60 percent. See McFarland et al. (2018). 


2 National Center for Public Policy and Higher Education 
(2011). 


3 In Ohio, only 15 percent of first-time, full-time, degree- 
seeking students at public two-year institutions earn de- 
grees within three years. See Ohio Department of Higher 
Education (2014). 


4 Weiss, Ratledge, Sommo, and Gupta (forthcoming). 


5 For more background on the origins of the demonstra- 
tion see Sommo and Ratledge (2016). For a fuller com- 
parison between the Ohio programs and CUNY ASAP, 
see Appendix B in the accompanying online supplemen- 
tary appendixes. 


6 While random assignment occurred at the college level, 
the main analyses in this brief pool outcomes across the 
three colleges. 


7 For more background on the origins of the demonstra- 
tion, see Sommo and Ratledge (2016). 


8 Degree in Three operates at two of Cuyahoga Commu- 
nity College’s four campuses. 


9 Nontraditional students are defined as those who were 
24 or older, who worked 35 or more hours per week, who 
had children, or who had not received a high school 
diploma and were not enrolled in high school at the time 
of random assignment. Nontraditional students are con- 
sidered to be at higher risk of not completing degrees. 


10 The final report will compare the services received by 
the program group and the control group. 


11 The data analysis does not account for summer enroll- 
ment or for tutoring required of students not performing 
well in their courses. It also does not account for the fact 
that the colleges eventually relaxed tutoring requirements 
for students who were receiving As or Bs in their courses 
at midterms. 


12 One college chose to allow students to spend the text- 
book funds before they used up their Pell Grant funds. As 
a result, students could get a refund for some Pell Grant 
funds they had not spent. The other colleges applied the 
textbook funds to student accounts after they exhausted 
their Pell funds, which meant those colleges may have 
spent less on textbooks. 


13 “Momentum” refers to the intensity at which stu- 
dents initially progress through college, for example, the 
number of credits students complete in their first year of 
college (including summer). Research has shown that 
academic momentum is positively correlated with degree 
completion. See Attewell, Heil, and Reisel (2012). 


14 Developmental credits are those earned by passing 
developmental courses. They do not count toward degree 
requirements. 
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15 To compare the current Ohio program results with 
those found in the CUNY ASAP study for a given outcome 
at a particular time after random assignment, one can 
calculate the difference between the estimated effects for 
each intervention. The standard error of the difference in 
effects is calculated as the square root of the sum of the 
squared standard error associated with each interven- 
tion’s estimated effect. 


16 This subgroup was prespecified as “confirmatory” in 
an analysis plan. See Schochet (2008). 


17 Cost data were collected from October 2014 through 
December 2017. 


18 CUNY ASAP’s costs have come down significantly over 
time as the program has evolved, implementing tiers of 
advising need similar to those used in Ohio and realizing 
efficiencies of scale. CUNY ASAP’s costs are now esti- 
mated to be approximately $3,400 per student per year. 


19 Hedges (2018). 
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Doubling Graduation Rates in a New State 
Two-Year Findings from the ASAP Ohio Demonstration 
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¢€ raduation rates more than doubled for program participants at three Ohio colleges that 
replicated the City University of New York (CUNY) Accelerated Study in Associate Programs 
(ASAP) program. ASAP requires students to enroll full time and provides comprehensive 


financial and academic support and other support services. This brief presents two-year impact, 
implementation, and cost findings on the ASAP demonstration in Ohio. The random assignment 
evaluation shows that students in the program group clearly outperformed the control group with respect 
to credit accumulation and graduation. Graduation rates more than doubled: 19 percent of the program 
group earned a degree or credential after two years compared with 8 percent of the control group. 
Additionally, most of the Ohio program components were well implemented, and the total annual direct 
cost per program group member was roughly $2,300, with some variation across colleges. Replications 
rarely deliver results comparable to those of original programs, but the Ohio demonstration has been an 
exception, achieving results similar to those of the original. 


